1. Field of the Invention
This invention relates to the conversion of noninterlaced video into an interlaced video format, and more particularly removing the flicker artifact resulting from noninterlaced video being encoded into interlaced video such as NTSC, PAL, SECAM, HDTV, or Super NTSC.
2. Related Art
While early computers relied on television set monitors to display computer graphics, television and computer graphics have since evolved differently. For instance, computer graphics are typically displayed in a noninterlaced format with a vertical frame refresh rate (frame rate) of 60-80 hertz (Hz), and encoded in the "Red-Green-Blue" (RGB) color space.
In contrast, standard video, such as used by televisions and VCRs, is interlaced video with a vertical frame refresh rate (frame rate) of 30 Hz (NTSC) or 25 Hz (PAL and SECAM), and is encoded in the YUV (PAL), the YIQ (NTSC), the YPrPb (HDTV), or the YCrCb color space.
The differences in standard video and computer graphics video did not present significant problems until the evolution of the "multi-media" environment where it is desirable to display both forms of video on the same television monitor. In order to display computer graphics on a television monitor or record it on a VCR, the noninterlaced computer graphics video must be encoded (converted) into an interlaced standard video format.
An interlaced video format uses two interleaved fields to display one frame of video. Each field is a series of video scan lines each representing a raster sweep across a monitor by an electron gun or other display device. Each scan line is further divided into individual picture elements (pixels).
Each of the two fields contains half of the scan lines of one frame of video. One of the fields in the frame contains the even numbered scan lines while the other field contains the odd numbered scan lines. Each frame of video is essentially one picture or "still" out of the series of pictures or "stills" which make up a video stream.
A "frame rate" is how often the video source repaints or "refreshes" the screen with a new video frame. In the NTSC system, the screen is refreshed every thirtieth of a second or at a frame rate of 30 Hz. For PAL, the frame rate is 25 Hz.
FIG. 1 represents an interlaced video format such as NTSC or PAL. A first field 100 includes even numbered scan lines 104, 106, 108, and 110. A second field 101 includes odd numbered scan lines 105, 107, 109 and 111. For a single video frame 103, field 100 is first scanned in direction 113 and displayed in its entirety. Then field 101 is scanned in direction 115 and displayed in its entirety. The combination of fields 100 and 101 makes up video frame 103 which includes scan lines 104-111. The process described above is then repeated for the next video frame 103.
Scan lines 104-111 are of course only representative of the hundreds of scan lines which make up a video frame. In a format such as NTSC, which has 525 scan lines per video frame 103, fields 100 and 101 have 262.5 scan lines each. For a format such as PAL or SECAM, which has 625 scan lines per video frame 103, each field 100 and 101 has 312.5 scan lines.
In contrast to standard television video, computer graphics typically use a noninterlaced format where all of the lines of a single video frame are scanned out sequentially. Consequently the term "field" is not applicable to noninterlaced computer graphics video.
Since in a noninterlaced format (in the computer graphics "domain") all of the scan lines making up a video frame are scanned out sequentially, a single frame of noninterlaced video contains twice the number of lines of either interlaced field 100 or 101. In order to display noninterlaced video on a television or record it on a VCR, fields must be derived from the noninterlaced frames and the data must be converted to an interlaced format such as NTSC, PAL, SECAM, digital composite, or digital component.
One relatively simple prior art method of converting noninterlaced video into interlaced video simply throws away every other scan line in each noninterlaced video frame to create a field. However, using this method, one-half of the vertical resolution for the resulting frame is discarded.
FIG. 2 shows another prior art conversion method. A noninterlaced video frame 200, including scan lines 204-211, is converted into a first interlaced field 200A including even scan lines 204, 206, 208 and 210. A second noninterlaced video frame 201 (which can be simply a second copy of noninterlaced video frame 200 from a buffer) including scan lines 212-219 is converted to a second interlaced field 201A including scan lines 213, 215, 217, and 219. Fields 200A and 201A are then scanned consecutively as discussed above. The result is interlaced video frame 203 which includes even scan lines 204, 206, 208 and 210 from noninterlaced video frame 200 and odd scan lines 213, 215, 217, and 219 from noninterlaced video frame 201.
The cost of implementing the two frame/two field method discussed above, and shown in FIG. 2, is minimal. However, an unfortunate side effect or artifact of this method is flickering. The flickering occurs because, as discussed above, in the computer graphics domain, the graphics or video data is typically displayed at a frame rate 70-80 noninterlaced frames per second, with approximately 65 microseconds between each scan line (see distance 230 in FIG. 2), while in the standard video domain, i.e., the NTSC, PAL, SECAM, or HDTV format, the video data is displayed at a frame rate of 30 (NTSC) or 25 (PAL and SECAM) interlaced frames per second with half the information, i.e. every odd line 213-219, being displayed for 16.67 milliseconds followed by the second half of the encoded video information, i.e., every even line 204-210, being displayed for 16.67 milliseconds. Consequently, a flicker is caused by a temporal discontinuity between the scan lines in the two formats. This problem is especially evident in motion video or graphics.
The effect of this temporal discontinuity is most evident when a scan line of the noninterlaced graphics video is of a dark luminance (Y) and an adjacent scan line is of a lighter luminance (Y), or vice-versa, i.e., when there is a large differential in the luminance between vertically adjacent pixels in the noninterlaced video. This large differential causes a flicker artifact oscillating at the field rate of the interlaced system (60 Hz for NTSC or 50 Hz for PAL or SECAM) and the larger the differential, the more visually noticeable the flicker.
There is a similar effect if one line of noninterlaced graphics data has a first value for one of the color components (U, I, V, Q, Cr, Cb, Pr, Pb, etc.) and a subsequent line has a second, different, value for one of these components, i.e., there is a large differential in one of the color components between vertically adjacent pixels. For example, in a checkerboard pattern of a first color and a second color, the resulting interlaced video frame will flicker first color, then second color, then first color, etc. at the field rate of 60 Hz (NTSC) or 50 Hz (PAL or SECAM). Consequently, there is an inherent potential for a flicker artifact for each video component whenever noninterlaced video is converted to an interlaced format by prior art methods.
One prior art solution to the flicker artifact is to line average, or interpolate, two or more scan lines of noninterlaced video into one scan line of interlaced video. FIG. 3 shows a prior art method of reducing the flicker artifact where three scan lines of noninterlaced video are line averaged to generate one scan line of interlaced video. In FIG. 3, a first noninterlaced video frame 300 is converted to an interlaced field 300A by: line averaging or weighing noninterlaced scan lines 304 and 305 to create interlaced scan line 320; line averaging noninterlaced scan lines 305, 306 and 307 to create interlaced scan line 322; line averaging noninterlaced scan lines 307, 308 and 309 to create interlaced scan line 324; and line averaging noninterlaced scan lines 309, 310 and 311 to create interlaced scan line 326.
As also seen in FIG. 3, noninterlaced video frame 301 (which can be simply a copy of video frame 300) is converted to a second interlaced field 301A by a similar line averaging method offset by one line.
Using the conventional "line averaging" method shown in FIG. 3, interlaced fields 300A and 301A are then combined by the frame interlace process shown in FIG. 1 to create interlaced video frame 303 including averaged scan lines 320 through 327. Interlaced video frame 303 can then be processed for use with a conventional television monitor or recorded on a VCR.
Using the line averaging method of FIG. 3, the effect of the hard vertical edge transitions described above is minimized because any large differential in the luminance, or other color component, of vertically adjacent pixels are smoothed out over several averaged interlaced scan lines. Therefore, the line averaging method of FIG. 3 does eliminate some of the flicker artifact. However, this line averaging method has several drawbacks.
First, this line averaging method darkens the resulting video frame 303 and blurs image edges. The blurring occurs on edges separating a pixel of a first luminance (Y) value from a pixel of a second luminance (Y) value, e.g., a dark pixel/light pixel transition that is one pixel wide. Using the line averaging method of FIG. 3, the luminance (Y) of both pixels are averaged and the contrast is lost. Thus, in the extreme situation of a black pixel/white pixel transition, a gray colored pixel is produced and the black/white contrast is lost.
Second, as discussed above, line averaging is only needed when there is a significant difference in the luminance, or one of the other color components, associated with vertically adjacent pixels of the noninterlaced video. However, with prior art line averaging methods the line averaging is performed on every scan line of every frame, whether or not there is a significant difference in the luminance, or color components, of the two vertically adjacent pixels. Therefore, with the prior art methods, even in frames where averaging is unnecessary, and there is no real potential for flicker, blurring and darkening is introduced without any real benefit. Further, with prior art methods, no way of correcting or compensating for the blurring and darkening effects of line averaging is provided. Additionally, prior art systems are highly inefficient and costly in terms of memory (circuitry) used.
In addition, with the prior art line averaging methods, care must be taken at the beginning and end of each noninterlaced video frame 300 and 301 because fewer scan lines are available for averaging. This complicates implementation.
More sophisticated versions of the line averaging method shown in FIG. 3 perform five and seven line averaging. However, as the number of scan lines to be averaged increases, so does the amount of darkening and blurring. In addition since, in the prior art, each scan line is averaged, as the number of lines to be averaged increases, the cost of the implementation increases.
Finally, as discussed above, conventional methods of reducing the flicker artifact, such the method shown in FIG. 3, typically only address the flicker associated with the luminance (Y) component of the interlaced video. Therefore, the inherent flicker associated with the other color components (U, I, V, Q, Cr, Cb, Pr, Pv, etc.) is still present.
The present inventors have recognized that what is needed is a method for correcting the flicker artifact associated with noninterlaced to interlaced video conversion which substantially removes the flicker artifact from all the color components but is more selective than prior art methods and does not compromise the color purity, brightness, resolution, or sharpness of the resulting interlaced display. The method should also be relatively inexpensive to implement, and be user programmable.